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Overview 


Many organizations use dashboards to monitor and analyze important performance measures. Data 
dashboards present the most important information (sometimes called key performance indicators, 
or KPls) about processes and outcomes in a concise, easy-to-digest format. Dashboards get their 
name from the clusters of instruments and gauges drivers use to monitor the behavior of their cars. 
As educational organizations invest in more comprehensive human capital management systems 
(HCMSs), they must track the implementation and effectiveness of these systems in order to realize 
their full benefits. A human capital management (HCM) dashboard provides a visual representation 
of the implementation and effectiveness of HCM processes related to recruiting, hiring, developing, 
and retaining effective educators. Such a dashboard can be used to track and monitor progress 
toward achieving HCM goals. This brief, which provides an overview of the development of HCMS 
dashboards, first describes some of the work that the U.S. Department of Education and Teacher 
Incentive Fund (TIF) districts have done to develop dashboards. It then presents an example of a 
comprehensive strategic dashboard and an example of metrics for a dashboard that focuses 
specifically on teacher recruitment. It concludes with a summary of 12 steps to consider in 
developing dashboards and a list of resources for developing HCM performance indicators that 
could be presented on a dashboard. 


Teacher & Leadership Programs 1 


IW ak-m Ohavid-xe eo) toh ksi-we ake lUler-hdlelam DY-I-valeler-) ae. 


The U.S. Department of Education has developed the United States Education Dashboard 
(http://dashboard.ed.gov/dashboard.aspx) to display information on a wide variety of education 
indicators, including some related to human capital. As shown in Figure 1, under the heading of 
“Teachers and Leaders” there is a count of how many states (as of 2010) have teacher and principal 
evaluation systems that involve student growth data, along with the percentage of teachers with a 
college major in their main assignment area (as of 2003-04 and 2007-08). This dashboard illustrates 
some of the key features of a dashboard, including simple, easy-to-read numeric summaries of 
important outcomes; historical trends, with arrows indicating improvement or decline; and drop- 
down options for “drilling down”! to see more detailed information, such as trend graphs, 
breakdowns by demographic characteristics, and state-level data. 


‘In the dashboard literature, to drill down means to move from a summary indicator (e.g., grantwide or 
districtwide average) to more detailed information (e.g., a breakout by teacher or student characteristics or by 
school). On a dashboard, a drilldown often involves clicking on a graph or a button to show the underlying data 
or choosing from a menu that takes the user to tables or graphs that provide a more detailed look at the 
indicator. 
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Figure 1. Screenshot of United States Education Dashboard 
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Figure 1. Screenshot of United States Education Dashboard—Continued 
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Dashboard Projects of Teacher 
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Federal initiatives such as TIF have encouraged grantees to collect more valid and more extensive 
information about educators, the key resource in improving student achievement and equitable 
access to effective instruction. Grantees have developed educator evaluation systems that provide 
better information on educator practices and their effects on student learning and have begun to 
use that information in making a wider range of HCM decisions. TIF has also stimulated additional 
data collection and analysis about educator recruitment, selection, professional development, and 
retention, as well as student access to effective educators. As these data accumulate, it can be 
challenging to turn it into actionable information for leaders, program managers, and human 
resource (HR) staff. This section presents examples of TIF grantees that have begun to develop 
dashboards to help district leaders, program managers, and other stakeholders make sense of the 
data and track the effectiveness of their grant-supported HCM initiatives. 


Austin Independent School District 


TIF 3 grantee Austin Independent School District has added a dashboard for principals to its teacher 
HCM data system. The district customized an off-the-shelf performance evaluation tracking and 
reporting tool to provide a simple dashboard for principals to use to track the progress of teacher 
evaluations being completed and to display and analyze patterns of results (such as which teachers 
are strong or weak on different performance dimensions, and which dimensions are strong or weak 
in the school). The dashboard provides heat map graphics to show schoolwide results by 
performance dimension. The district also provides a link to professional development options 
aligned to each performance dimension on the summary page for teacher observations, so that 
principals have a place to start in planning schoolwide or individual professional development. 
Austin’s experience illustrates two important principles of dashboard development: (1) keeping the 
dashboard simple to avoid overburdening users and to focus attention on actionable metrics, and 
(2) pilot-testing the dashboard with intended users. Austin has also found that dashboard use 
increases with principal experience. First-year principals have so much to learn that they make little 
use of even this relatively focused tool. However, by their third year, principals have become 
comfortable enough to use the tool's full functionality. 


Pittsburgh Public Schools 


TIF 2 and 3 grantee Pittsburgh Public Schools has a long-term commitment to developing and 
enhancing HCM dashboards, originating from the district's participation in the Gates Foundation’s 
Empowering Effective Teachers initiative. Currently the district has dashboards for teacher 
effectiveness and has developed several staffing metrics that are dashboard-ready. The teacher 
effectiveness dashboard presents information such as the proportion of teachers at different levels 
of performance on each of its teacher effectiveness measures (practice, student achievement 
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growth, and student survey results, how the proportion varies over time, and by whether teachers 
are newly hired and in high-needs schools. The dashboard also tracks the performance of teachers 
retained and whether low-performing teachers improved or exited the district. Another indicator 
included is the share of students who experienced instruction by high- and low-performing teachers 
in the last school year. The staffing metrics track the proportion of teachers that are new to a school 
in that year and the ratio of internal applicants to vacancies. These are used indicators of how 
attractive the school is to the existing teaching workforce during the district's site-based selection 
process, and are used to determine whether a school should receive additional staffing support 
from the district central office. The district is considering adding an external applicant-to-vacancy 
ratio and the average time to fill vacancies. District staff emphasized the importance of selecting 
actionable data and indicators, telling a coherent story related to a specific issue or need rather than 
including all of the information available. 


Denver Public Schools 


While not exactly a dashboard, the Strategic Data Project has developed a set of human capital 
diagnostics for Denver Public Schools. Their report, available at 
http://cepr.harvard.edu/files/cepr/files/sdp-diagnostic-hc-dps.pdf, includes charts and graphs 
describing a variety of human capital outcomes, including information on bonus payouts, the 
relationship between payouts and student growth, teacher retention by student growth, and 
characteristics of schools teachers transferred into and out of. Many of these graphs could be 
regularly updated and presented as a dashboard to allow program administrators and district 
leaders to track important outputs and outcomes related to the performance-based compensation 
system. 


Texas Region 18 Education Service Center 


As part of their TIF 4-supported work, Region 18 has begun developing a comprehensive HCM data 
system that will include data dashboards for a variety of users, including the superintendents of 
participating districts, staff with HCM responsibilities, principals, and teachers. Region 18 began their 
journey toward this comprehensive system with the need to track the components of their multiple- 
measure evaluation processes for teachers and principals for earlier TIF grants. As they worked with 
the disparate data systems of districts participating in the grant, Region 18 staff saw the need to 
develop an integrated system and the potential to build data dashboards for different users. They 
engaged multiple stakeholders, including superintendents, principals, teachers, and HR staff, in the 
development of the request for proposals to build the integrated data system. Their envisioned 
system will be based on six modules: talent acquisition, training and professional development, 
educator evaluation, student growth, educator effectiveness, and performance-based 
compensation. The talent acquisition module would contain information on applicant qualifications, 
hiring, and retention. The training and professional development module is intended to allow 
educators to create PD plans, provide a catalog of PD offerings, record PD activities, and track PD 
participation. The educator evaluation module will help teachers and administrators manage the 
evaluation of practice, calculate evaluation ratings, and report results. The student growth module 
would capture information related to value-added, student learning objectives, and student 
portfolios, enabling teachers to create SLOs and monitor student progress, and provide teacher- and 
school-level summaries of results. The educator effectiveness module would calculate an overall 
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effectiveness rating from multiple performance measures and provide reports, while the 
performance-based compensation module would calculate individuals’ performance-based 
compensation based on performance measures and provide reports. Each set of users will have a 
dashboard designed to present summary information, and district- and educator-level profiles will 


also be available. 
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When considering building a comprehensive HCM dashboard, it is important to recognize that 
different users and audiences are often interested in different performance indicators. Typically, 
district leaders are concerned with broad outcomes that provide a picture of the overall state of 
human capital in the district, as well as progress toward key HCM goals, whereas HR staff may be 
more interested in measures of how well specific HR processes are working and their costs. The 
dashboard literature refers to dashboards that are designed to present an overall picture of top- 
level indicators as strategic dashboards, while those aimed at tracking specific processes are called 
operational dashboards. 


Figure 2 shows a portion of a strategic HCM dashboard for district leaders that focuses on educator 
recruitment and selection. Since district leaders are responsible for a wider range of outcomes, their 
dashboard should focus only on the crucial results of HCM processes, those that are critical to 
meeting broader district goals for student achievement and equity. Thus, the dashboard shows only 
a few recruitment and selection indicators, so that the whole HCM dashboard remains relatively 
simple and easy to follow. 


Figure 2. Recruitment and Selection Panel of HCM Dashboard for District Leaders 
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As shown in Figure 2, the dashboard presents three key indicators of how well the district’s 
recruitment and selection processes are delivering a key resource—qualified and effective 
teachers—to the district and its schools. The first panel, showing the trend in the percentage of 
vacant teaching positions at the beginning of the school year, tracks the ability of the HCM process 
to fill positions on time. The second panel, showing the percentage of positions filled with 
substitutes or teachers teaching outside their field of certification, tracks how well the HCM system 
staffs schools with teachers prepared to teach their subjects. The third panel reflects the quality of 
the hires made, by tracking the proportion of those hired in each year who are later rated as 
effective or higher on their performance. Note that the choice of two years after the hire as the 
reference period for performance reflects the possibility that teachers may need time to develop 
their skills. Some districts may want to consider performance in the initial year or at the end of a 
probationary period. Though this dashboard includes only three indicators on the main panel, it 
could be designed to allow the viewer to drill down to the school level or other levels of interest in 
the district, such as by subject or grade, school performance level, or level of student poverty. 


Other key metrics, beyond those for staffing, could focus on other HCM practice areas and 
important HCM outcomes such as teacher performance and turnover. Table 1 shows examples of 
possible metrics for the teaching workforce: new teacher induction (onboarding), mentoring, 
professional development, performance management (teacher evaluation), compensation, and 
turnover. 


Table 1. Potential Metrics for District Leader HCM Dashboard 


Human capital management arte 
F g Potential metric or key performance indicator 


function 
: Percent of new teachers attending orientation program 
Induction ; 
Percent of new teachers assigned a peer coach for first year 
e Percent of teachers with a mentor assigned 
Mentoring 


Average teacher rating of the effectiveness of their mentor 


Percent of teachers participating in professional 
development activities linked to their evaluation results 
Percent of teachers participating in a professional learning 
community 


Professional development 


Percent of teachers receiving annual evaluation on time 
Performance management Percent of teachers rated effective or highly effective 
Percent of teachers meeting student learning objectives 


Percent of new teachers receiving hiring bonus 
Average hiring bonus 

Compensation Percent of teachers receiving performance pay 
Salary as a percent of total operating budget 
Benefits as a percent of total operating budget 


Retention rate of effective and highly effective teachers 


Turnover/retention Turnover rate of ineffective teachers 
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Data on the metrics in Table 1 could be collected and reported over multiple time periods, just as for 
the staffing indicators shown in Figure 1. Also, the dashboard could provide drilldowns or breakouts 
of the data to allow more nuanced analysis and interpretation of the data. Examples of drilldowns 
include school/grade level, teacher type (e.g., science, technology, engineering, and mathematics; 
special education; teacher leader), teacher performance and experience levels, and school 
performance level. 
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While HCM program administrators are interested in the district leader HCM metrics and their 
drilldowns, they may find many other specific metrics to be useful for assessing the functioning and 
cost of specific HR practice areas. Therefore, it would be beneficial for the district to establish HCM 
metrics and dashboards that would represent KPls for various HCM practice areas. The system 
would display these dashboards within the HR department, though the dashboards could be shared 
out when necessary. 


The teacher staffing practice area again can be used to illustrate creation of a specific staffing 
dashboard. In the example below, there are five KPls: applicants/vacancy, time to fill a vacancy, 
rating of overall new hire quality, percentage of applicants from underrepresented groups, and cost 
per hire. There is also one critical drilldown: the recruitment source from which each applicant was 
recruited. Table 2 displays the information for this staffing dashboard, along with some hypothetical 
data, for a given year. These data would be collected and reported over multiple years as well. 


Table 2. Metrics/Key Performance Indicators for Staffing Dashboard 
Metric/key performance indicator 
Percent of 
applicants 
Application- Average __| from under- 
to-vacancy | Time to fill | quality of | represented | Cost per 
Recruitment source ratio (Days) applicant groups hire 
Internal transfer 3.1 45 4.0 8% $1,100 
Dike ce CO macivith 1.0 35 4.9 15% $2,700 
applicant 
Job fair 4.2 50 3.3 25% $5,600 
Print ads 2.1 54 2.6 22% $3,400 
Internet ads 25.7 37 3.9 33% $3,100 
Social networks 10.9 40 2.8 20% $2,600 
Employee referrals 1.4 38 4.8 30% $1,500 
Teacher preparation 
program placement 38.8 55 4.4 25% $5,800 
offices 
fonsSidns Histol 15 59 40 40% $1,500 
teaching 
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An inspection of the data in Table 2 shows that for any KPI, there are large differences among the 
various recruitment sources. So how might the HR leader use these data? A major use would be in 
the development of a staffing plan for the next hiring cycle. Based on their performance, certain 
recruitment sources could possibly be dropped or modified to improve their performance. For 
example, due to job fairs’ subpar performance as a recruitment source, HR leaders might decide to 
discontinue job fairs altogether, or hold fewer of them (thus reducing total cost), hold them in new 
locations where there is likely to be more applicant interest (thus increasing applicants/reducing 
vacancy), or be more selective at the job fair itself in determining which attendees to interview 
further. 


Staff could construct similar practice-area specific dashboards for each of the other HCM areas, 
including induction and mentoring, professional development, performance management, and 
turnover/retention. 
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Steps in Developing an HCMS Dashboard 


Since dashboards are widely used in many sectors for performance management and reports, there 
are many books and internet resources available that provide guidance on various aspects of 
dashboard development. We reviewed a wide range of these resources and have distilled the 
process down to the 12 steps described in this section. 


1. Define the audience for the dashboard and its purposes. As mentioned above, strategic 
dashboards, often designed for district leaders, would present different performance indicators or 
metrics than operational dashboards designed for HR staff or school leaders. Top leaders are 
responsible for successful implementation of strategy. Their dashboard should focus on a 
relatively small set of HOM measures that are most directly relevant to achieving the district's 
strategic goals. HR staff need a broader and deeper view. They need to track the key systems that 
drive the HOM measures. 


2. Involve potential users in the development process. Involving other users early helps to surface 
potential problems with usability as well as to ensure that metrics or indicators are credible, 
relevant, and actionable. It also provides a communications channel about the project and 
develops a base of potential supporters. 


3. lect the metrics (also called key performance indicators, or KP!s). KP!s should represent factors 
that the users can influence or control (are ‘actionable”), are related to the purposes and interests 
of the intended audience, and are based on valid data that are obtainable at reasonable cost. 
Limit the number of metrics or KPIs to be included. Many resources recommend 5 to 10, and 
rarely more than 15 for a strategic dashboard. Information on additional metrics could be 
provided by linking some of the KP!s to more detailed operational dashboards that track the 
processes that produce the results. To keep the dashboard usable, do not include something as a 
KP! just because you have the data. Box 71 provides some suggestions for ways to begin developing 
kP!s and some resources for identifying KPIs. 


4. Determine where data underlying the KPIs will come from, how they will be collected, and how 
often they will be updated. Much KP! data will come from existing data systems, but some 
important KP!s may require new data collection systems or changes to current data collection 
routines. For example, to track the effectiveness of recruitment efforts, you may want to assess the 
guality of applicants for vacant teaching positions. This could require developing and 
administering a survey of principals that asks them to assess the quality of those they interview. 
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5. Decide if you want the dashboard to communicate goals and progress toward goals. One 
important purpose of dashboards Is to monitor the status of HOV processes or the progress 
toward HCV goals. To make progress clear, dashboards can include data from earlier periods and 
indicate trends over time. Operational dashboards often use colors or graphics, like traffic lights, 
to show viewers whether processes are going well, trending in the wrong direction, or are well 
below targets or expectations. 


6. Decide what drilldowns or disaggregations will be presented. Designers should consult with users 
to decide what details users need to see. Disaggregation of KP!s by individual school or types of 
schools (e.g., high needs or low performing versus others) should be considered. To tell the story, 
designers should consider what context factors might differentiate or confound patterns of results 
and allow the indicators to be segmented by those factors. Drilldowns are also useful to reinforce 
accountability for results at lower levels, such as the school or central office department/team, 
Just as school report cards are deigned to focus attention on individual schools. Drilldowns can 
also be used to connect operational dashboards to strategic dashboards, allowing users to trace 
the results reported in the latter to the processes that determine them. 


7. Layout the dashboard as a prototype. While details of design are beyond the scope of this brief, 
some basic principles can be stated briefly: 


e Organize the dashboard so that the first view users see is the highest level of the indicators of 
interest to the intended users. More detailed information can be presented using drilldowns or 
menus, as exemplified by the United States Education Dashboard shown in Figure 7. 


e Donot crowd. A frequent recommendation is to show four to seven indicators per page, which 
allows ‘at a glance” evaluation. Provide more information via drilldowns, accessible by 
clicking on or hovering over the table or graphic or by using a simple menu. 


e Include trends or goals to provide context for interpreting whether the numbers reported are 
good or bad. 


e Decide whether and how to present alerts if trends are going in the wrong direction or the 
latest results are far from goals. Some dashboards use colors (e.g., “traffic light” colors: green 
for positive trends of measures that met standards or goals, yellow for a trend moving in the 
wrong direction, red for measures that are far below standards or goals). 


e Though an attractive design is important, do not emphasize design over getting the KPI's right. 


8. Have users review prototypes and evaluate the completeness and relevance of the KPIs, the 
understandability of tables and graphs, and the usability of the information presented. Do the 
menus and drilldowns work?Are the data that are presented correct?Are the data updated as 
intended? Revise and test again before going live. 


9. Determine who will have access to the dashboard, and especially what drilldown information 
different users will be able to access. One consideration here is maintaining appropriate levels of 
confidentiality for publically accessible dashboards. Disaggregating data to small units can 
compromise confidentiality. 


10. Develop documentation that describes the data and decisions behind the dashboard. As the 
dashboard evolves and the developers move on to other projects, it is important to document 
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11. 


12. 


what data go into the KP!s, where the data come from, how often they are updated, why the KP! 
was chosen for presentation, how the indicator was calculated, and any cautions on the validity or 
interpretation of the data. 


Foll out, communicate purpose, train, and monitor usage. Inform potential users who have not 
been part of the development process about what the dashboard contains and train them on how 
to access, navigate, and interpret the information it provides, especially if trends are shown. It 

may also be necessary to produce a short video clip showing how the dashboard works and how 
to use it to monitor, analyze, and understand HCV processes and outcomes. After the dashboard 
goes live, monitor who accesses it and what menu options or drilldowns are used to get a better 
sense of whom the audience really is, and identify KPls that are not of interest and could be 
replaced. 


Be prepared to change after the dashboard goes live. Experts suggest that once dashboards are 
up and running and users begin to use the data for program management, they will come up with 
additional indicators and breakouts or find some indicators less useful. Plan for a review of the 
dashboard with a sample of users after the initial rollout. Review the dashboard annually 
thereafter. Some indicators may become less important, and new ones may emerge as the district 
and its HCMS evolve. 
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This brief provided an overview of the purpose, scope, and key decisions that need to be made to 
begin to develop an HCM dashboard. The 12 steps presented also apply to just about any 
dashboard project. If developing a comprehensive HCM dashboard appears daunting, it is also 
possible to start small, concentrating at first on one or two HCM activities. Given the central 
importance of assessing educator performance, TIF and TSL grantees might consider beginning with 
strategic dashboards that report on performance distributions, equitable access to effective 
educators, and the relationship between performance and compensation. Another natural starting 
point may be operational dashboards that track process completeness (e.g., the number of 
educators observed the required number of times, the proportion receiving written feedback or a 
post-conference) and evaluation results use (e.g., the number engaging in professional development 
related to areas in need of improvement, percentage of teachers rated less than effective who 
improved their performance in the subsequent year). 


Whether starting small or aiming to be comprehensive, dashboards have great potential to help 
program managers understand whether specific components of the HCMS are being implemented 
as intended, and whether the HCM initiatives are succeeding in attracting, developing, and retaining 
educators with the competencies to achieve broader goals for equity and student learning. A 
dashboard can also help communicate about the project to district leaders and key stakeholders. 
Additionally, constructing a dashboard can be another opportunity for TIF and TSL grantees to think 
through what specific HCM results they want their grant-supported activities to accomplish. 
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